Errors and secret data in the Italian research assessment exercise. A comment to a reply

نویسندگان

  • Alberto Baccini
  • Giuseppe De Nicolao
چکیده

Italy adopted a performance-based system for funding universities that is centered on the results of a national research assessment exercise, realized by a governmental agency (ANVUR). ANVUR evaluated papers by using “a dual system of evaluation”, that is by informed peer review or by bibliometrics. In view of validating that system, ANVUR performed an experiment for estimating the agreement between informed review and bibliometrics. (Ancaiani et al. 2015) presents the main results of the experiment. (Alberto Baccini and De Nicolao 2017) documented in a letter, among other critical issues, that the statistical analysis was not realized on a random sample of articles. A reply to the letter has been published by Research Evaluation (Benedetto et al. 2017). This note highlights that in the reply there are (1) errors in data, (2) problems with “representativeness” of the sample, (3) unverifiable claims about weights used for calculating kappas, (4) undisclosed averaging procedures; (5) a statement about “same protocol in all areas” contradicted by official reports. Last but not least: the data used by the authors continue to be undisclosed. A general warning concludes: many recently published papers use data originating from Italian research assessment exercise. These data are not accessible to the scientific community and consequently these papers are not reproducible. They can be hardly considered as containing sound evidence at least until authors or ANVUR disclose the data necessary for replication.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Interregional Patient Mobility in the Italian NHS: A Case of Badly-Managed Decentralization; Comment on “Regional Incentives and Patient Cross-Border Mobility: Evidence From the Italian Experience”

The article by Brenna and Spandonaro on interregional mobility for acute hospital care in Italy raises important issues concerning social and territorial equity in a healthcare system. Based on Regions and private providers’ strategic behavior, the hypothesis adopted to explain patient cross-border mobility (CBM), demonstrated by statistical analysis, may be further explored using qualitative m...

متن کامل

Radically Questioning the Principle of the Least Restrictive Alternative: A Reply to Nir Eyal; Comment on “Nudging by Shaming, Shaming by Nudging”

In his insightful editorial, Nir Eyal explores the connections between nudging and shaming. One upshot of his argument is that we should question the principle of the least restrictive alternative in public health and health policy. In this commentary, I maintain that Eyal’s argument undermines only a rather implausible version of the principle of the least restrictive alternative and I sketch ...

متن کامل

The Health Mobility Is All True Mobility?; Comment on “Regional Incentives and Patient Cross-border Mobility: Evidence From the Italian Experience”

In their study, Brenna and Spandonaro analyzed the mobility into Italian regions. In particular, it analyzes the situation of 5 regions, with very different backgrounds. With this paper, we try to better define the meaning of health mobility and to find its underlying causes. Furthermore, we propose a strategy that could help in controlling mobility flows that currently are the source of health...

متن کامل

Environmental gamma radiation: a comment (Letter to the Editor)

Editor, I read the recent publication by Toossi et al. with a great interest (1). Toossi et al. concluded that “Average gonad and bone marrow doses for North Khorasan, Boshehr and Hormozgan provinces were less than the corresponding values for normal area (2).” There are some facts on this report to be concerned. I agree that the detected levels might be high in the mentioned area, but th...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:
  • CoRR

دوره abs/1707.06822  شماره 

صفحات  -

تاریخ انتشار 2017